Regularization Parameter Selection for Faulty Neural Networks

Regularization techniques have attracted many researches in the past decades. Most focus on designing the regularization term, and few on the optimal regularization parameter selection, especially for faulty neural networks. As is known that in the real world, the node faults often inevitably take place, which would lead to many faulty network patterns. If employing the conventional method, i.e., the test set method or cross-validation, to find the optimal regularization parameter, it will cost a lot of time. Moreover, although some statistic methods have been proposed, almost of them aim at the fault-free networks. Thus, in the paper, a MPE formula is derived to evaluate the mean prediction error in the multi-node open fault situations and then used to select the optimal regularization parameter. Experiment results have shown that the optimal parameter value selected by our proposed formula is very close to the actual one, chosen by the conventional test set method. Our contribution is that our proposed MPE formula can be used in choosing the regularization parameter instead of the test set method for faulty neural networks with multi-node open fault. Keywords—Faulty neural networks, multi-node open fault, regularization parameter, mean prediction error.